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16.  Abstract 

This  study  was  designed  to  determine  the  relationships  among  experimental  tests,  the 
tests  currently  used  to  select  entrants  into  the  FAA's  Air  Traffic  Control  Specialist 
(ATCS)  Academy  Nonradar  Screening  Program,  and  Academy  success.  A  battery  of 
paper-and-penci 1  tests,  including  the  Directional  Headings  Test  (DHT)  which  was 
developed  at  the  Civil  Aeromedical  Inst5tute->  and  subsequently  modified  for  this 
study,  and  the  Dial  Reading  Test  (DRT)  was  administered  to  1,255  students  who  entered 
in  1987.  Scores  on  the  current  selection  measures,  the  Multiplex  Controller  Aptitude 
Test  (MCAT)  and  Abstract  Reasoning  Test,  (ART),' were  obtained  from  the  Office  of 
Personnel  Management  (0PM). 

The  pass,  fail,  and  withdrawal  rates  for  the  sample  were  compared  to  perfonnance 
levels  on  the  experimental  and  0PM  tests.  Several  Academy  performance  criteria  and 
test  results  were  intercorrelated  and  stepwise  multiple  regression  analyses  were 
conducted  to  predict  overall  Academy  success  and  final  grades. 

Results  demonstrated  that  the  DHT  and  DRT  could  be  used  to  assess  the  potential  of 
entrants  to  successfully  complete  the  Academy  Screen.  Even  though  the  current  ATCS 
applicant  group  differs  from  those  entering  the  Academy  a  decade  ago  on  several 
dimensions  (e.g.,  education  and  experience),  the  multiple  correlation  of  the  DHT, 
DRT,  and  MCAT  scores  with  Academy  success  remained  relatively  unchanged.  Results 
suggest  thatthere  are  other  test  measures  which,  when  combined  with  the  MCAT,  would 
be  better  predictors  than  the  existing  battery  of  selection  tests. 
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CORRELATES  OF  TWO  EXPERIMENTAL  TESTS 
WITH  PERFORMANCE  IN  THE  FAA  ACADEMY 
AIR  TRAFFIC  CONTROL  NONRADAR  SCREEN  PROGRAM 


INTRODUCTION 

The  use  of  a  battery  of  Civil  Service  Commission  (CSC)  aptitude  tests  for  Air  Traffic 
Control  (ATC)  selection  was  initiated  in  1962.  Initially,  the  CSC  battery  was  required  only 
for  qualifying  individuals  without  previous  experience:  however  in  1964,  the  battery  was 
required  for  all  applicants,  regardless  of  experience  (Boone,  1979).  With  some  minor 
modifications,  this  selection  procedure  remained  in  place  through  August,  1981.  The  current 
selection  battery,  implemented  in  1981,  is  comprised  of  two  aptitude  tests,  the  Multiplex 
Controller  Aptitude  Test  (MCAT)  and  the  Abstract  Reasoning  Test  (ART),  and  a  job 
knowledge  test,  the  Occupational  Knowledge  Test  (OKT).  Currently,  the  OPM  battery  is  part  of 
a  two-stage  selection  process.  Following  qualification  on  the  paper-and-pencil  OPM  battery, 
completion  of  an  interview,  and  medical  and  security  evaluations,  applicants  are  hired  as 
FAA  employees.  They  then  enter  the  second  stage  of  the  selection  process,  a  nine  week 
performance-based  Screen  program  (i.e.,  pass/fail)  at  the  FAA  Academy  in  Oklahoma  City. 

During  the  past  few  years,  the  FAA  has  sought  to  develop  more  extensive  recruitment 
techniques  and  more  efficient  procedures  for  testing.  Interviewing,  and  selecting  applicants 
into  the  Academy  program.  As  part  of  this  effort.  Interest  has  been  expressed  in  the 
availability  of  shorter  test  Instruments  that  could  be  administered  during  recruitment 
efforts  to  identify  individuals  who  possess  high  potential  for  success  as  Air  Traffic  Control 
Specialists  (ATCSs).  This  study  was  designed  to  determine  the  relationships  of  two  shorter 
instruments,  the  Directional  Headings  Test  (DHT)  and  Dial  Reading  Test  (DRT),  with 
performance  on  tests  in  the  current  selection  battery.  These  tests  were  used  in  previous 
Civil  Aeromedical  Institute  (CAMI)  investigations  by  Cobb  and  Matthews  (1972)  and  Boone 
(1979)  prior  to  the  implementation  of  the  current  selection  battery.  Additionally,  results  of 
the  comparisons  will  be  used  to  determine  the  utility  of  the  two  tests,  along  with  tests  from 
the  current  selection  battery,  in  improving  the  effectiveness  of  the  selection  process. 

METHODS 

Subjects. 

In  1987,  the  DHT  and  DRT  were  administered  as  part  of  a  battery  of  experimental  tests  to 
1,255  students  who  entered  the  FAA  Academy  Nonradar  Screen  Program.  Entrants  who  were 
repeating  the  program  or  those  who  cdme  from  special  emphasis  programs  were  excluded 
from  the  sample.  Women  comprised  12.1%  and  minority  group  members  comprised  6.3%  of 
this  sample. 

Measures. 

OPM  Battery.  This  battery  is  administered  to  applicants  to  determine  if  they  qualify  for 
entry  into  the  Academy  Nonradar  Screen  Program.  The  MCAT  Is  a  timed  test  that  requires 
the  Individual  to  respond  to  a  series  of  questions  by  utilizing  a)  tabular  data  about  aircraft 
altitude,  speed,  and  course,  along  with  b)  a  map  which  contains  several  aircraft  at  different 
locations  on  various  flight  paths.  This  information  is  used  to  compute  aircraft  tlme-and- 
distance  travel  patterns  and  to  determine  if  there  are  potential  conflicts  In  the  flight  paths. 
The  ART  is  an  older,  timed  Civil  Service  Commission  test  (OPM- 157)  in  which  applicants 
are  presented  a  series  of  figures  or  letters,  and  are  then  asked  to  select  the  choice  that 
carries  out  a  principle  of  logical  development.  The  OKT  contains  questions  about  the 
phraseology  and  procedures  used  by  airmen  and  ATCSs. 

Weights  are  applied  to  the  raw  scores  on  the  MCAT  and  ART,  to  produce  a  distribution  of 
transformed  scores  with  a  mean  of  70  and  a  maximum  score  of  100.  The  Individual's 
transformed  score  is  referred  to  as  the  Transmuted  Composite  (TMC).  The  final  OPM  rating 
includes  the  addition  of  extra  credit  points  based  on  their  OKT  scores  and  Veteran's 


preference  points.  OPM  battery  performance  measures  for  this  study  Included  scores  on  the 
MCAT.  AIRT.  and  TMC,  but  excluded  the  final  OPM  rating. 

DHT.  Ihe  DHT  was  developed  by  Cobb  and  Matthews  (1972)  as  a  paper-and-pencil  measure 
of  speeded  spatial  ability  that  informal  Job  analyses  indicated  was  a  major  determinant  of 
ATC  performance.  Originally,  It  was  comprised  of  three  separate  parts.  Subjects  were  given 
90  seconds  to  complete  each  part  of  the  test.  Each  item  was  comprised  of  one,  two,  or  three 
bits  of  information  that  reflected  the  cardinal  points  on  a  mariner's  compass.  For 
example,  the  letter  "E",  the  symbol  "  -♦  ",  and  the  notation  "90"  each  denoted  "East."  Other 
combinations  of  letters,  symbols,  and  degrees  denoted  "West",  "North",  and  "South."  In  Part  1 
of  the  test,  the  subject  rapidly  determined  if  the  Information  conflicted  or  agreed.  The  item 
was  followed  by  one  of  five  questions:  North?,  East?.  West?,  South?,  or  Conflict?,  to  which 
a  yes  or  no  response  was  given.  Part  II  was  similar  to  Part  I  except  the  subject  answered 
whether  the  presented  data  represented  the  exact  opposite  of  the  given  heading.  Part  111 
involved  the  addition  of  distraction  to  Part  II  questions.  Boone  (1979)  indicated  that  (he 
DHT  was  not  considered  for  inclusion  in  the  selection  battery  since  the  field  testing 
facilities  might  be  unable  to  Insure  that  the  strict  timing  and  control  requirements  would  be 
implemented. 

Using  the  basic  principles  of  the  DHT,  modified  procedures  were  developed  that  permitted 
expansion  of  the  testing  time  to  5  minutes  for  each  of  two  parts.  Each  part  of  the  DHT  now 
contains  120  items.  In  each  problem,  subjects  are  presented  with  three  bits  of  information 
relating  to  the  cardinal  points  of  a  compass.  In  Part  I  of  the  test  the  subject  must  rapidly 
scan  the  available  Information  and  determine  either  the  direction  indicated  (if  all 
information  is  consistent),  or  indicate  that  the  information  is  inconsistent.  In  Part  II,  the 
task  is  somewhat  more  complicated.  The  determination  of  direction  can  be  established  if  at 
least  two  of  the  three  types  of  information  are  consistent.  The  inconsistent  alternative  is 
selected  only  when  all  three  bits  of  information  disagree. 

DRT.  The  DRT  was  developed  in  the  1950’s  by  the  USAF  Air  Training  Command  at 
Lackland  Air  Force  Base  for  use  in  the  selection  of  candidates  for  undergraduate  pilot 
training.  Subjects  are  presented  with  seven  different  Instrument  dials  for  each  set  of 
questions.  The  questions  require  the  individual  to  Identify  and  correctly  read  the 
appropriate  dial  in  order  to  select  the  correct  answer  from  among  five  choices.  The  allotted 
time  for  completion  of  the  57  questions  is  11.5  minutes.  This  test  has  been  used  in  previous 
research  with  ATCS  subjects  (Boone  1979,  Marshall-Mies  and  Colmen  1976,  and  Brokaw 
1957).  Tests  scores  have  been  found  to  be  predictive  of  training  performance  in  studies  with 
samples  of  navigators,  ATCSs,  and  undergraduate  pilots. 

Criterion  Measures. 

The  Academy  ATCS  Nonradar  Screen  Program  assesses  trainees'  ability  to  learn  a  set  of 
air  traffic  procedures  and  then  to  apply  those  procedures  during  a  series  of  nonradar, 
laboratory  simulations.  The  performance  measures,  with  their  percentage  of  the  final  score, 
include:  academic  tests  (20%);  a  paper-and-pencil  Controller  Skills  Test  (20%);  and  two 
evaluations  of  performance  on  the  best  five  of  six  graded  laboratory  problems  -  an 
Instructor  assessment  (30%)  and  a  technical  assessment  (30%).  A  final  grade  of  70.0  is 
required  for  successful  completion  of  the  ATCS  Nonradar  Screen.  The  final  status  of  an 
entrant  can  be  pass,  fail,  or  withdrawal  before  completion  of  the  program. 

Analyses. 

Four  series  of  analyses  were  conducted  in  this  study: 

1)  Descriptive  statistics  for  various  components  of  the  experimental  tests  and  OPM  tests 
were  computed: 

2)  The  relationships  of  overall  Academy  success  (measured  by  pass.  fail,  and  withdrawal 
rates)  to  the  experimental  and  OPM  test  scores  were  examined; 
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3)  Intercorrelations  of  test  results  and  various  Academy  performance  criteria  were 
calculated;  and 

4)  Stepwise  multiple  regression  analyses  were  performed,  using  corrected  correlation 
coefficients  to  predict  Academy  performance. 

Correlations  between  the  experimental  and  OPM  tests  with  the  criterion  measures 
(Academy  performance)  were  spuriously  low.  due  to  the  well-known  restriction-in-range 
effect  (Thorndike.  1949).  Correlation  coefficients  for  the  selected  sample  were  lower  than 
would  have  been  the  case  if  the  entire  applicant  group  had  entered  the  Academy  Nonradar 
Screen.  This  occurs  as  a  consequence  of  the  reduction  in  the  range  and  variances  of  the  test 
scores  in  the  selected  sample.  Coefficients  were  corrected  for  this  restriction  using  the  two- 
variable  equation  for  the  OPM  tests  The  three-variable  equation  was  employed  for  the  DHT 
and  DRT.  since  variances  for  those  tests  were  not  available  for  the  ATCS  applicant 
population. 

RESULTS  AND  DISCUSSION 

Table  I  presents  descriptive  statistics  for  various  measures  of  the  two  tests  for  the  1.255 
Academy  entrants  with  valid  (non-missing)  data  on  both  tests.  For  the  Directional  Headings 
Test,  the  mean  of  correct  item  responses  on  Part  II  was  57.2,  clearly  lower  than  the  mean  of 
correct  answers  for  Part  I  of  72.2.  Also,  no  one  achieved  the  maximum  possible  score  of  120 
on  Part  II,  as  had  occurred  on  Part  I.  Since  (he  number  of  correct  responses  was  so  different 
for  the  two  DHT  parts  and  in  order  to  compensate  for  possible  guessing  on  the  test  items. 
DHT  total  scores  were  adjusted.  The  measure  of  combined  performance  for  both  parts  of  the 
DHT  in  subsequent  analyses  was  based  on  the  product  of  the  number  of  correct  items  and 
the  percentage  of  correct  items  for  Part  1,  plus  the  result  of  the  same  formula  applied  to  Part 
II. 

The  mean  correct  score  on  the  Dial  Reading  Test  was  43.7,  compared  to  the  maximum 
possible  total  of  57.  Results  of  a  t-test  Indicated  that  this  value  was  higher  (p<.C)01)  than  the 
40.97  average  noted  in  the  Boone  (1979)  study.  An  adjusted  measure  of  performance  on  the 
DRT  was  computed  in  a  similar  manner  as  was  done  with  the  DHT.  Specifically,  the 
measure  of  DRT  performance  used  in  this  study  was  represented  by  the  product  of  the 
number  of  correct  DRT  items  and  the  percentage  of  correct  items. 


TABLE  1.  Descriptive  Statistics  for  DHT  and  DRT 


Mean 

SD 

Minimum 

Maximum 

Test  Measure 

72.2 

15.4 

23.0 

120.0 

Number  correct  -  DHT  Part  I 

57.2 

13.6 

6.0 

103.0 

Number  correct  -  DHT  Part  11 

129.4 

24.6 

33.0 

207.0 

Total  correct  -  Parts  1  &  II 

125.1 

25.7 

13.2 

204.0 

DHT  adjusted  Parts  I  &  II 

43.7 

8.0 

9.0 

57.0 

Total  correct  -  DRT 

38.5 

10.0 

2.7 

57.0 

DRT  adjusted 

Oist 


ar 


Directional  Headinfts  Test  and  Academy  Success.  The  sample  was  divided  into  quartiles 
and  the  pass,  fail,  and  withdrawal  rates  (percentages)  for  the  groups  were  examined  using  a 
chi-square  statistic  (Table  2).  Pass  rates  ranged  from  39.6  for  those  with  the  lowest  DHT  jn_ 
scores  to  69.3  for  the  highest  DHT  quartlle.  The  chi-square  comparison  of  these  differences  — 
was  statistically  significant  (p<.001).  The  percentages  of  trainees  failing  the  Screen  program 
or  withdrawing  before  completion  of  the  program  were  both  Inversely  related  to  DHT  ~ 
scores.  The  failure  rate  was  43.8%  for  the  lowest  DHT  quartile  compared  to  25.3%  for  the 
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highest  quartile  of  DHT  scores.  Similarly.  16.6%  of  those  in  the  lowest  quartile  withdrew, 
compared  to  only  5.4%  in  the  highest  quartile. 


The  final  Academy  scores  for  the  DHT  groups  were  compared  with  one-way  analyses  of 
variance  and  the  Scheffe  test  was  used  for  post-hoc  comparisons  (Table  2).  The  one-way 
analysis  of  variance  revealed  a  statistically  significant  difference  (p<.001)  in  the  average 
grades  of  students  based  on  their  DHT  scores.  Compared  to  the  average  of  67.5  for  students 
in  the  lowest  quartile  of  the  DHT,  those  in  the  highest  quartile  had  an  average  score  of  74.5. 
Group  comparisons,  using  the  Scheffe  test,  revealed  that  the  average  grade  of  the  fourth 
quartile  students  differed  significantly  (d<.01)  from  that  of  students  in  the  first  two 
quartlles.  Also,  the  third  quartile  group  had  a  higher  Academy  final  grade  average  than  did 
the  lowest  quartile  group. 

TABLE  2.  DHT  with  Academy  Success  and  Academy  Final  Score. 

ACADEMY  PASS.  FAIL.  WITHDRAWAL  RATES 


Group 

N 

Pass 

Fall 

Withdraw 

Quartile  1 

313 

39.6 

43.8 

16.6 

Quartile  2 

314 

51.3 

38.9 

9.8 

Quartile  3 

315 

61.3 

29.5 

9.2 

Quartile  4 

313 

69.3 

25.3 

5.4 

TOTAL 

1255 

55.4 

34.3 

10.3 

CHI-SQUARE 

62.6** 

29.9** 

22.0** 

(D.F.=3) 

ACADEMY  FINAL  SCORE 


Group 

N* 

Mean 

SD 

Quartile  1 

261 

67.5 

11.8 

Quartile  2 

283 

70.5 

10.2 

Quartile  3 

286 

72.8 

10.4 

Quartile  4 

296 

74.5 

10.2 

TOTAL 

1126 

71.5 

11.0 

•  Ebccludes  those  who  withdrew  before  completion  of  the  Screen  program. 

'  **  p<.001  F  ratio  =  22.34** 

Dial  Reading  Test  and  Academy  Success.  The  sample  was  divided  into  quartlles  based  on 
DF?T  performance.  Pass,  fail,  and  withdrawal  rates  for  the  groups  were  then  examined,  using 
a  chi-square  statistic  (Table  3).  Differences  in  pass  rates  based  on  the  DRT  were  similar  to 
those  noted  for  the  DHT.  The  quartile  of  the  sample  with  a  DRT  score  of  46.2  or  higher  had  a 
pass  rate  of  66.3%,  compared  to  a  pass  rate  for  the  lowest  quartile  of  40.1%.  Failure  rates 
were  Inversely  related  to  DRT  performance  with  almost  one-half  (48.4%)  of  the  lowest 
quartile  group  falling  the  Screen.  Withdrawal  rates  did  not  differ  significantly  between  the 
groups,  although  only  7%  of  the  students  in  the  top  quartile  withdrew,  compared  with  10.0% 
to  12.5%  for  the  other  quartlles. 

Final  Academy  grades  were  also  compared,  using  a  one-way  analysis  of  variance  and 
Scheffe  tests  of  significance  between  group  means  (Table  3).  Differences  in  Academy  grades 
for  the  DRT  performance  groups  reflected  the  differences  found  in  pass  rates.  Mean  grades 
ranged  from  67.6  for  the  first  quartile  to  74.5  for  the  fourth  quartile.  As  was  the  case  with 
the  DHT.  the  mean  Academy  final  score  of  74.5  for  the  highest  DRT  group  was  significantly 
higher  (p<.01)  than  the  Academy  scores  of  the  lowest  two  quartlles  of  DRT  performance. 
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Also,  the  group  of  Academy  entrants  In  the  third  quartlle  of  DRT  performance  had 
significantly  higher  final  scores  than  those  in  the  first  quartlle. 


TABLE  3.  DRT  with  Academy  Success  and  Academy  Final  Score. 
ACADEMY  PASS.  FAIL.  AND  WITHDRAWAL  RATES 


Group 

N 

Pass 

Fail 

Withdraw 

Quartlle  1 

312 

40.1 

48.4 

11.5 

Quartlle  2 

312 

54.2 

33.3 

12.5 

Quartlle  3 

319 

60.8 

29.2 

10.0 

Quartlle  4 

312 

66.3 

26.7 

7.0 

TOTAL 

1255 

55.4 

34.3 

10.3 

48.8** 

39.5** 

5.8 

(D.F.=3) 


ACADEMY  FINAL  SCORE 


Group 

N* 

Mean 

SD 

Quartlle  1 

276 

67.6 

11.5 

Quartlle  2 

273 

70.7 

10.2 

Quartlle  3 

287 

72.8 

10.3 

Quartlle  4 

290 

74.5 

10.6 

TOTAL 

1126 

71.5 

11.0 

*  Excludes  those  who  withdrew  before  completion  of  the  Screen  program. 

**  p<.001  F  ratlo=  21.64  ** 

Current  Selection  Measures  and  Academy  Success.  The  sample  was  also  divided  into 
quartlles  based  on  Transmuted  Composite  scores  (TMC  -  used  as  the  primary  selection 
measure  of  applicants  for  the  Job  of  air  traffic  controller).  The  quartlle  of  the  sample  with 
TMC  scores  of  93.7  or  greater  had  a  pass  rate  of  66.1%,  compared  to  the  pass  rate  of  45.4% 
for  the  lowest  quartlle  (TMC  scores  of  87.7  and  lower).  Lower  rates  of  Academy  failure  and 
withdrawal  were  evidenced  as  TMC  scores  Increased  (Table  4). 

The  mean  Academy  score  of  74.2  for  trainees  in  the  top  quartlle  of  the  TMC  was 
significantly  higher  than  that  for  trainees  in  each  of  the  other  TMC  groups.  None  of  the 
remaining  comparisons  reached  statistical  significance. 
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TABLE  4.  OPM  Selection  Test  Composite  (TMC)  with  Academy  Success 

and  Academy  Final  Score. 

ACADEMY  PASS.  FAIL.  AND  WITHDRAWAL  RATES 


Group 

N 

Pass 

Fall 

Withdraw 

Quartile  1 

315 

45.4 

40.0 

14.6 

Quartile  2 

301 

55.1 

32.9 

12.0 

Quartile  3 

320 

54.7 

37.2 

8.1 

Quartile  4 

319 

66.1 

27.3 

6.6 

TOTAL 

1255 

55.4 

34.3 

10.3 

CHI-SQUARE 

27.7*** 

13.0** 

13.6** 

(D.F.=3) 


ACADEMY  FINAL  SCORE 


Group 

N* 

Mean 

SD 

Quartile  1 

269 

69.7 

11.1 

Quartile  2 

265 

71.1 

10.3 

Quartile  3 

294 

70.7 

11.2 

Quartlle  4 

298 

74.2 

10.6 

TOTAL 

1126 

71.5 

11.0 

*  Excludes  those  who  withdrew  before  completion  of  the  Screen  program. 

**  p<.01 

***  p<.001  F  ratio  =  9.25*** 

One  possible  use  of  the  DHT  and  DRT  would  be  to  employ  these  tests  in  conjunction  with 
current  selection  measures.  Table  5  shows  the  Academy  pass  rates  for  the  various  DHT  and 
DRT  performance  levels  (based  on  quartlle  values  for  the  total  sample)  for  the  315  trainees 
in  the  lowest  quartile  of  TMC  scores  (87.7  or  lower).  Within  this  group,  where  the  average 
pass  rate  was  45.4%,  scores  on  both  the  DHT  and  DRT  test  differentially  predicted  Academy 
success.  Of  the  trainees  who  scored  in  the  upper  quartlle  of  the  DHT,  68.8%  successfully 
completed  the  Academy  Nonradar  Screen  Program.  A  similar  percentage  (65.6%)  of  those 
with  higher  scores  on  the  DRT  were  successful.  In  contrast,  of  those  trainees  in  the  lower 
quartiles  of  the  DHT,  only  29.7%  and  39.0%  were  successful.  Percentages  of  successful 
trainees  in  the  two  lower  quartiles  of  the  DRT  were  34.4%  and  48.0%,  respectively.  While  the 
utility  of  using  the  DHT  or  DRT  to  predict  the  successful  Academy  performance  of  trainees 
with  lower  scores  on  the  OPM  battery  needs  to  be  cross-validated  with  another  sample, 
results  suggest  that  the  two  tests  measure  somewhat  different  aspects  of  trainee  aptitudes 
than  do  the  OPM  scores. 

TABLE  5.  DHT  and  DRT  Performance  and  Academy  Success  for  the  Lowest 

Quartlle  of  OPM  Test  Scores. 


Group 

DHT 

DRT 

N 

%  Pass 

N 

%  Pass 

Quartlle  1 

111 

29.7 

128 

34.4 

Quartlle  2 

77 

39.0 

100 

48.0 

Quartlle  3 

79 

59.5 

55 

54.5 

Quartlle  4 

48 

68.8 

32 

65.6 

TOTAL 

315 

45.4 

315 

45.4 
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Differential  Performance  on  Experimental  and  PPM  Tests.  The  distribution  of  the  scores 
of  men  and  women  on  the  DHT,  DRT,  and  TMC  did  not  differ  significantly.  Comparisons  of 
the  distributions  for  minorities  and  nonminorities  revealed  a  significant  difference  on  the 
DRT  and  TMC,  but  not  the  DHT.  Of  minorities  in  the  sample,  48. 1%  had  DRT  scores  that  fell 
in  the  lowest  quartile.  A  higher  percentage  (39.2%)  of  the  minorities  also  had  TMC  scores 
that  fell  within  the  lowest  quartile.  While  these  differences  do  not  necessarily  mean  that 
there  would  be  corresponding  differences  in  the  distributions  of  the  scores  between 
respective  applicant  subgroups,  previous  research  by  Rock,  et  al,  (1984)  did  demonstrate  that 
women  and  minorities  scored  lower  than  men  and  nonminorities  respectively,  on  an 
experimental  version  of  the  existing  OPM  ATCS  test  battery. 

Interrelationships  Between  Test  Results  and  Academy  Performance.  First,  the  Academy 
entrants'  scores  on  the  experimental  tests  and  the  scores  on  the  OPM  selection  tests  were 
correlated  using  the  point-blserlal  method  with  the  dichotomous  criterion  measure  of 
Academy  pass/not  pass.  Also,  the  matrix  of  Pearson  correlation  coefficients  between  the 
various  tests  is  presented  in  Table  6.  The  DRT  and  DHT  exhibited  a  higher  correlation  with 
each  other  (.45)  than  with  any  of  the  other  tests.  The  TMC  had  a  higher  correlation  with  the 
DRT  than  the  DHT  (.33  and  .24,  respectively).  Following  corrections  for  restrlctlon-in- 
range,  the  TMC  displayed  the  highest  correlation  (.46)  with  the  pass/not  pass  criterion 
measure.  The  corrected  correlation  of  .20  between  the  criterion  measure  and  the  APJT  was 
the  smallest. 


TABLE  6.  Matrix  of  Intercorrelations*  of  Experimental  Tests,  OPM  Tests. 

and  Academy  Success. 


DHT 

DRT 

ART 

MCAT 

TMC 

DHT 

DRT 

.45 

... 

ART 

.17 

.26 

... 

MCAT 

.21 

.27 

.12 

... 

TMC 

.24 

.33 

.49 

.92 

... 

PASS/NOT  PASS 

.23 

.23 

.12 

.14 

.17 

PASS /NOT  PASS 

(Corrected  for  range 
restriction) 

.41 

.44 

.20 

.38 

.46 

*  COEFFICIENTS  OF  .  10  OR  GREATER  (p<.001) 

Next,  experimental  test  scores  and  the  OPM  selection  measures  were  correlated  with 
several  measures  of  Academy  performance  previously  mentioned,  including  overall  final 
score  (NLCOMP),  academic  tests  (NLBA  and  NLCPT),  average  technical  assessment  (AVTA), 
average  instructor  assessment  (AVIA),  average  score  on  laboratory  problems  (AVL5).  and  the 
Controller  Skills  Test  (CST)  (Table  7).  The  sample  was  now  necessarily  limited  to  those 
trainees  who  had  completed  (passed  or  failed;  excluding  those  who  withdrew)  the  Academy 
program.  Once  again,  comparisons  of  the  various  tests  and  the  criterion  measures  were 
corrected  for  restrlction-in-range.  Following  corrections,  correlations  between  the  tests  and 
the  overall  Academy  grade  (NLCOMP)  were  similar  to  those  noted  for  the  pass/not  pass 
criterion.  Of  the  various  component  measures  of  the  Academy  grades,  the  experimental  tests 
and  OPM  tests  were  most  closely  correlated  with  performance  on  the  CST  (ranging  from  .32 
with  the  ART  to  .56  with  the  TMC  and  DRT).  Correlations  with  the  two  measures  of 
laboratory  performance  (AVTA  and  AVIA)  were  very  similar.  Performance  on  the  two 
academic  tests  (NLBA  and  NLCPT)  exhibited  relatively  low  correlations  with  the  various 
tests  (ranging  from  .03  for  the  NLBA  with  the  MCAT  to  .26  for  the  DHT  and  TMC  with  the 
NLCPT).  Correlations  between  the  ART  and  various  measures  of  Academy  performance  were, 
v/ith  few  exceptions,  lower  than  those  of  the  other  predictor  tests. 
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TABLE  7.  Correlations*  of  Experimental  Tests  and  OPM  tests  with 
Academy  Performance  Components. 


NLCOMP 

NLBA 

NLCPT 

AVL5 

AVTA 

AVIA 

CST 

DHT 

26(47) 

17(23) 

16(26) 

22(38) 

22(39) 

23(38) 

28(49) 

DRT 

29(52) 

09(18) 

12(25) 

22(42) 

21(41) 

20(39) 

32(56) 

MCAT 

17(45) 

01(03) 

13(36) 

13(36) 

12(33) 

16(43) 

ART 

16(26) 

13(21) 

10(16) 

10(16) 

11(18) 

09(15) 

20(32) 

TMC 

21(54) 

06(18) 

09(26) 

15(41) 

16(44) 

14(39) 

22(56) 

N  =  1 126;  COEFFICIENTS  OF  1 1  OR  GREATER  {p<.001) 

•  DECIMAL  POINTS  OF  COEFFICIENTS  HAVE  BEEN  DELETED. 

(  )  VALIDITY  COEFFICIENTS  AFTER  ADJUSTMENT  FOR  RESTRICTION-IN-RANGE 


Table  8  presents  the  results  of  multiple  regression  analyses  which  used  the  OPM  and 
experimental  tests  as  predictors,  and  Academy  success  and  Academy  final  scores  as  criteria. 
A  stepwise  method  was  used  in  which  a  test  measure  was  entered  Into  a  given  model  only  if 
it  met  the  significance  criterion  of  p<.01.  The  ART  was  not  a  significant  predictor  of 
Academy  success  or  final  score  when  combined  with  the  experimental  tests,  or  with  the 
experimental  tests  plus  the  MCAT.  The  MCAT.  combined  with  the  two  experimental  tests, 
was  a  significant  measure,  but  had  lower  beta  weights  than  either  the  DHT  or  the  DRT. 

TABLE  8.  Multiple  Regression  Analyses  of  Validity  Coefficients  of  Test  Scores 

and  Academy  Performance  Criteria. 


CRITERION  -  PASS/NOT  PASS  ACADEMY  SCREEN 


MODEL  BfULTIPLE  R  |  BETA  WEIGHT 


1)  DRT  and  DHT  .469 


DRT  .30 


DHT  .21 


2)  DRT.  DHT.  and  ART  .469 


DRT  .30 


DHT  .21 


3)  DRT.  DHT.  MCAT.  and  ART  .481 


DRT  .23 


DHT  .19 


MCAT  .14 


CRITERION  •  ACADEMY  FINAL  SCORE 


MODEL  MULTIPLE  R  BETA  WEIGHT 


1)  DRT  and  DHT  .548 


DRT  .37 


DHT  .23 


2)  DRT.  DHT.  and  ART  .548 


DRT  .37 


DHT  I  .23 


3)  DRT.  DHT.  MCAT.  and  ART  .563 


DRT  .29 


DHT  .20 


MCAT  .17 


Multiple  regression  models,  which  included  the  MCAT,  DHT,  and  DRT  produced  multiple 
correlations  of  .481  (pass/not  pass)  and  .563  (final  grade),  were  similar  to  those  found  by 
Boone  (1979),  with  the  exception  of  the  beta  weights  for  the  three  tests.  Boone  (1979) 
performed  several  multiple  regression  analyses  in  order  to  compare  various  models 
comprised  of  the  then  current  CSC  selection  tests,  the  now  current  selection  tests,  the  DRT, 
and  the  earlier  version  of  the  DHT.  His  results  for  a  model  containing  the  MCAT,  DHT,  and 
DRT  as  predictors  had  a  multiple  correlation  of  .566  with  the  Academy  lab  score.  The  beta 
weights  were  MCAT  (.31).  DHT  (.14),  and  DRT  (.19). 

CONCLUSIONS 

Results  demonstrate  that  the  DRT  and  the  DHT,  each  requiring  under  15  minutes  of 
administration  time,  can  separately,  or  in  combination  be  used  to  assess  the  potential  of 
individuals  to  successfully  complete  the  FAA  Academy  Nonradar  Screen  Program.  The 
application  of  these  tests  during  recruitment  to  provide  feedback  to  individuals  concerning 
their  potential  for  the  air  traffic  profession  should  be  undertaken  only  after  careful 
consideration  and  following  cross-validation  of  the  results  of  the  present  study. 

A  significant  concern  associated  with  this  application  would  be  the  tendency  for 
recruitment  personnel  to  use  the  test  results  as  ’’pre-selection"  information  and  discourage 
those  with  low  DHT  and  DRT  scores  from  applying  to  take  the  OPM  selection  battery. 
Careful  training  of  recruitment  personnel  in  the  appropriate  use  of  the  test  data  would  be 
required.  Additionally,  there  are  other  techniques  available  to  assist  in  the  recruitment  of 
potential  applicants  that  may  be  less  time-consuming  than  the  use  of  special  tests.  Research 
by  Collins.  Nye,  and  Manning  (1990)  evaluated  the  use  of  biographical  data  as  one  means  of 
identifying  applicants  who  have  high  potential  for  successfully  completing  the  Academy 
Nonradar  Screen  Program. 

Even  though  the  current  ATCS  applicant  group  differs  significantly  from  groups  entering 
the  Academy  a  decade  ago  on  selected  dimensions  (e.g.,  education  and  experience),  the  utility 
of  the  DHT,  DRT,  and  MCAT  In  predicting  Academy  Screen  success  remains  relatively 
unchanged.  These  results  do  Indicate  that  there  are  other  aptitude  measures  that,  when 
added  to  the  MCAT.  could  prove  to  be  better  predictors  than  the  existing  battery. 

Further  research,  using  a  greater  variety  of  aptitude  measures  is  needed  to  more  closely 
identify  the  combination  of  those  measures  that  best  predicts  performance  in  the  Academy 
Nonradar  Screen  Program.  Determinations  of  the  validity  and  utility  of  the  various 
measures  must  also  be  expanded  to  include  other  criterion  measures,  such  as  on-the-job 
training  performance. 
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